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1.  Objective 


The  objective  of  this  research  study  is  to  design  and  experiment  with  an  interface 
that  is  tailored  to  individual  Soldiers’  cognitive  styles,  individual  differences,  and 
expertise  while  at  the  same  time  reduces  the  interface  complexity  perceived  by  the 
Soldier.  This  interface  design  will  improve  upon  existing  adaptive  interfaces  by 
going  beyond  adaptation  to  the  individual’s  prior  actions  and  tailoring  the 
interface  to  how  each  user  perceives,  processes,  and  filters  information  without 
the  added  complexity  of  current  adaptive  interfaces.  The  field  of  adaptive  user 
interfaces  (UIs)  and  human-computer  interaction  will  be  extended  by  this  tailored 
interface  innovation  to  investigate  Soldier  performance  benefits  of  speed, 
accuracy,  understanding,  coordination,  and  the  reduction  of  workload. 

2.  Introduction/Approach 

The  tactical  and  operational  environment  for  the  Army  is  changing  with  an  ever¬ 
growing  emphasis  and  need  for  information.  This  has  created  information- 
overload  challenges  for  the  Soldier.  Most  information  and  Soldier  systems  are 
viewed  through  a  computer  interface,  but  these  interfaces  are  typically  ignored  or 
not  considered  as  a  vital  component  in  the  Soldier  system.  There  is  a  large  amount 
of  diversity  in  how  people  perceive,  store,  and  process  information.  This  creates  a 
considerable  design  challenge,  as  most  interfaces  are  designed  to  be  one-size-fits- 
all.  Instead  of  ignoring  this  diversity,  there  is  great  potential  in  understanding  and 
capitalizing  this  diversity  if  the  way  an  interface  organizes  and  presents 
information  could  be  tailored  to  each  individual  and  their  own  cognitive  style. 
Past  attempts  of  adaptive  UIs  have  tried  to  monitor  and/or  predict  the  actions  of 
users  and  change  the  interface  based  on  past  user  actions  or  predictions  of  future 
actions.  These  interfaces  have  been  largely  unsuccessful  due  to  the  lack  of 
reliability  in  user  predictions  and  creating  an  ever-changing  interface  that  forces 
the  user  to  constantly  leam  and  understand  what  the  system  is  doing,  especially  in 
dynamic  environments  like  the  Army’s.  This  research  effort  attempts  to 
understand,  measure,  and  design  a  system  to  tailor  itself  according  to  the 
information  processing  strengths  and  weaknesses  of  each  individual  user  and 
Soldier. 

The  overall  approach  for  this  project  can  be  summarized  in  3  parts:  1)  the 
redesign  of  the  interface  to  the  unmanned  aerial  vehicle  (UAV)  task,  2) 
development  of  a  new  approach  to  tailor  the  interface  to  an  individual’s  cognitive 
style,  and  3)  software  development  to  support  human  performance 
experimentation  for  this  new  approach.  The  first  effort  was  not  initially  planned 
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but  became  a  requirement  at  the  initial  stages  of  the  project  when  it  was 
discovered  through  the  task  analysis  efforts  that  the  current  UI  was  not  designed 
for  the  current  UAV  operator’s  task. 

2.1  Interface  Redesign 

Early  versions  of  UAV  systems  required  manual  control  of  the  aircraft,  which 
required  pilot  training  and  expertise.  As  the  Army  UAV  systems  became  more 
autonomous,  the  UAV  controls  transformed  from  stick  and  rudder  input  devices 
to  “point  and  click”  navigation  controls  with  a  mouse  and  a  map.  Piloting  skills 
were  no  longer  required,  and  the  task  transformed  itself  from  a  piloting  task  to  a 
supervisory  control  task  for  UAV  operators,  where  their  primary  task  was  to 
monitor  the  system  for  abnormal  situations  and  intervene  as  necessary.  Even 
though  the  task  itself  had  transformed  and  the  controls  had  evolved,  the  UI  and 
the  way  information  was  presented  to  the  operator  had  not  significantly  changed. 
The  current  UI  still  presents  information  to  the  operators  as  if  they  were  the  UAV 
pilots  from  the  past  using  legacy  aviation  symbology  and  information.  Because  of 
this  discovery,  the  negative  feedback  that  the  current  UAV  UI  was  receiving  from 
its  operators,  and  the  overall  poor  design  of  the  current  UI,  I  decided  to  redesign 
the  UAV  UI.  The  redesign  was  completely  independent  of  the  current  and  legacy 
systems  and  was  driven  specifically  by  the  task  and  informational  requirements  of 
the  UAV  operator. 

2.2  Cognitively  Tailored  Interface 

Once  the  UI  was  adequately  tailored  to  the  task,  the  second  effort  was  to  continue 
tailoring  the  information  by  customizing  it  to  each  individual  operator.  This 
approach  extends  the  traditional  human  systems  integration  (HSI)  approach  and 
takes  adaptive  interfaces  into  a  different  direction  than  past  research.  Traditional 
HSI  approaches  typically  stop  tailoring  information  or  the  system  at  the  task  level 
mentioned  in  the  previous  subsection  and  do  not  attempt  to  consider  the  cognitive 
strengths  and  weaknesses  of  the  individual.  One  reason  for  this  is  the  difficulty  of 
considering  and  designing  the  diversity  and  variability  among  potential  users, 
which  has  been  identified  in  Psychology  and  Neuropsychology.  Ojemann  and 
Schoenfield- McNeill  (1999),  for  example,  showed  at  the  neural  level  that  no  2 
people’s  brains  store  the  same  information  in  the  same  way  or  in  the  same  place. 
The  individual  differences  field  within  psychology  has  identified  dozens  of 
variables  that  people  reliably  differ  across,  including  motivation,  intelligence, 
verbal  and  spatial  abilities,  and  cognitive  styles  (e.g.,  Maltby  et  al.  2007).  The 
challenge  of  enhancing  a  Soldier’s  individual  cognitive  strengths  and 
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compensating  for  their  individual  weaknesses  should  be  embraced  and  not 
ignored  by  researchers  and  system  architects. 

Adaptive  UIs  (e.g.,  Norcio  and  Stanley  1989)  have  been  one  approach  to  address 
the  user  diversity  and  individual  differences  design  challenge.  Adaptive  interfaces 
change  their  displays  and  available  actions  based  on  the  interpreted  user’s  goals 
and  past  actions.  There  has  been  some  success  with  adaptive  interfaces  (e.g., 
Sears  and  Shneiderman  1994,  Findlater  and  Mcgrenere  2008),  but  there  are 
considerable  costs  associated  with  them  as  well.  One  of  the  primary  disadvantages 
of  these  systems  is  that  the  interface  is  constantly  changing,  which  impedes 
learning  with  repeated  use  of  the  system,  especially  when  the  adaptive  interface  is 
not  accurate  in  its  changes  (Greenberg  and  Witten  1985).  Adaptive  interfaces  also 
place  an  extra  cognitive  burden  on  the  user  to  understand  the  behaviors  and 
operations  of  the  system  in  addition  to  adding  complexity  to  an  already  complex 
information  environment  for  the  Soldier  (Letsu-Dake  and  Ntuen  2009). 

The  approach  taken  with  this  research  was  to  identify  cognitive  attributes  that 
were  stable  over  time  and  provided  insight  on  how  each  operator  individually 
processed  information.  Cognitive  styles  (Riding  and  Rayner  2007)  were  selected 
as  this  attribute  to  tailor  the  information  within  the  UI.  Cognitive  styles  have  been 
found  to  exist  across  2  orthogonal  dimensions:  a  wholistic-analytic  and  the 
verbal-imagery  dimension.  The  wholistic-analytic  cognitive  style  continuum 
describes  how  an  individual  organizes  and  structures  information.  At  the  wholistic 
end  of  the  continuum,  there  is  a  tendency  and  preference  to  organize  and  structure 
information  at  the  global  level  with  an  emphasis  on  the  big  picture.  At  the  analytic 
end  of  the  continuum,  however,  there  is  a  tendency  to  organize  and  process 
information  at  a  detailed  level  of  the  individual  components.  Past  research  in 
classroom  settings  has  shown  significant  performance  improvement  when  the 
information  presented  to  the  students  matches  their  individual  cognitive  styles 
(e.g.,  Riding  and  Cheema  1991). 

Experimenting  with  the  wholistic-analytic  cognitive  style  is  meant  to  be  the  first 
of  many  types  of  cognitive  attributes  to  which  information  can  be  tailored.  Other 
specific  attributes  that  have  been  identified  for  future  research  efforts  beyond  this 
project  are  the  imagery-verbal  cognitive  style  and  an  individual’s  level  of 
expertise  for  the  task.  Future  efforts  will  not  only  look  at  the  impact  of  each  of 
these  other  attributes  but  also  begin  to  explore  the  interactions  across  these 
attributes.  The  research  approach  is  summarized  in  Fig.  1  to  show  how  a 
repository  of  promising  attributes  are  tested  and  then  integrated  with  each  other  to 
provide  the  largest  positive  impact  to  the  Soldier. 
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Fig.  1  Research  cycle  to  study  and  build  a  repository  of  cognitive  skills  and  attributes  for 
a  cognitively  tailored  interface 

2.3  UAV  Experimental  Environment 

A  synthetic  task  environment  (STE)  for  UAV  operations  was  developed  to  collect 
human  performance  measurements  using  the  cognitively  tailored  interfaces 
(CTIs).  The  STE  was  based  on  Wickens  and  Dixon’s  (2002)  UAV  simulation  that 
captures  the  3  core  components  of  UAV  operations,  1)  navigation,  2)  visual 
search  for  targets,  and  3)  monitor  for  system  failures,  and  has  enough  control  to 
be  used  for  repeated  experimentation.  The  STE  has  a  completely  custom  UI  that 
can  be  manipulated  to  support  the  different  cognitive  styles  and  expertise  designs. 
The  STE  can  simulate  multiple  short  UAV  missions  and  also  has  human 
performance  data  collection  capabilities  to  collect  navigation,  target  search, 
system  monitoring,  and  situation  awareness  data.  Finally,  the  STE  has  the  built-in 
capability  to  assess  the  individuals’  cognitive  style  through  a  short  cognitive 
styles  assessment  and  tailor  the  interface  based  on  their  own  specific  cognitive 
style  score. 

The  system  tailors  the  interface  to  the  person’s  cognitive  style  by  dividing  the 
wholistic-analytic  continuum  into  3  equal  categories  to  represent  1)  wholistic,  2) 
hybrid,  and  3)  analytic  preferences.  A  UI  was  designed  for  each  of  the  3 
categories,  and  based  on  their  cognitive  style  assessment,  which  is  done  in  real 
time,  the  system  selects  and  presents  to  them  the  UI  that  matches  their  cognitive 
style.  The  system  and  UI  do  not  change,  therefore  eliminating  the  problems  of  a 
constantly  changing  interface  that  past  adaptive  interfaces  have  encountered.  In 
order  to  test  and  refine  these  designs  before  they  are  used  with  the  UAV  STE, 
another  experiment  and  assessment  environment  was  created  to  test  how  well  the 
design  facilitates  performance  for  the  style  of  person  it  was  designed  for.  The 
results  for  these  3  project  areas  are  summarized  in  the  following  section. 
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3.  Results 


3.1  Interface  Redesign 

A  cognitive  task  and  work  analysis  combined  with  past  research  and  interviews 
with  UAV  operators  identified  that,  fundamentally,  controlling  the  UAV  is  now 
primarily  a  supervisory  control  task  given  the  high  level  of  automation  in  the 
system.  The  primary  requirement  for  the  operator  is  to  monitor  the  state  of  the 
system  for  emergencies  or  alerts  that  need  the  operator’s  attention  (Wickens  and 
Dixon  2002).  The  current  ground  control  station  UI,  however,  still  presents 
information  as  if  the  UAV  operators  were  the  pilots  that  used  the  system  in  the 
past.  Essentially,  the  current  UI  is  designed  for  UAV  pilots  of  the  past  and  not  the 
UAV  operators  of  today  or  tomorrow. 

The  task  analysis  confirmed  the  general  negative  feedback  from  interviewed  UAV 
operators  related  to  the  current  UI  for  the  universal  ground  control  stations  (Fig. 
2).  The  task  analysis  revealed  that  the  interface  contained  no  cohesive  design, 
requiring  the  operator  to  figure  out  the  appropriate  layout  of  more  than  a  dozen 
individual  windows  and  screens.  Among  this  collection  of  windows,  important 
alerts  and  warnings  can  be  quickly  and  easily  lost  behind  other  windows. 


Fig.  2  An  example  of  a  current  interface  for  unmanned  aerial  vehicle  operators 
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The  menu  hierarchy  for  the  system  is  both  wide  and  deep,  requiring  that  the  user 
spend  unnecessary  time  and  effort  remembering  where  needed  functions  and 
controls  exist  in  the  UI  (Fig.  3).  These  significant  UI  issues,  with  the  lack  of 
support  for  the  system  as  a  supervisory  control  task,  were  the  motivation  to 
redesign  the  system  based  on  the  current  UAV  operator’s  needs  and  requirements. 


Fig.  3  A  portion  of  the  menu  hierarchy  for  a  current  unmanned  aerial  vehicle  interface 

In  addition  to  identifying  the  3  core  UAV  tasks  mentioned  previously  (navigation, 
visual  search  for  targets,  and  monitoring  the  system)  and  the  supervisory  control 
nature  of  the  task,  through  task  analysis  and  past  research  we  identified  6  critical 
categories  of  information  UAV  operators  need  to  complete  their  mission  (Drury  et 
al.  2006): 

•  3-dimensional  (3-D)  spatial  relationships  (e.g.,  UAV  location  to  terrain) 

•  Weather  near  the  UAV 

•  Health  of  the  UAV  (e.g.,  warning  and  alerts) 

•  Status  of  the  UAV  (e.g.,  landing  gear  state) 

•  Operational  threats 

•  Mission-related  information 
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These  information  categories  guided  what  needed  to  be  visualized  and  the  general 
organization  of  that  information.  Each  visualization  went  through  several 
iterations  to  produce  a  dashboard  interface  design.  A  dashboard  is  a  visual  display 
that  consolidates  the  most  important  information  on  a  single  screen  so  the  system 
can  be  monitored  at  a  glance.  These  visualizations  address  the  6  content  areas 
mentioned  above  and  were  constructed  to  take  advantage  of  preattentive 
processing,  which  is  the  early  stage  of  visual  perception  that  rapidly  occurs  below 
the  level  of  consciousness.  These  preattentive  visual  attributes  include 
organization,  color,  position,  form  (orientation,  enclosures,  proximity),  and 
motion  for  rapid  perception  (Ware  2013). 


Several  data  visualization  inventions  from  the  leading  researchers  in  the  field 
were  used,  along  with  new  visual  concepts  that  I  invented.  Bullet  (Few  2006)  and 
sparkline  (Tufte  2006)  graphs  were  used  to  show  the  status  of  several  aircraft 
parameters  (Fig.  4).  These  graphs  allow  a  lot  of  information  to  be  summarized  in 
a  small  amount  of  space  and  monitored  quickly  by  the  operator.  The  sparkline 
graphs  display  past  performance  for  each  parameter  over  the  last  2  h,  which 
provides  a  history  and  context  to  the  current  value  that  is  not  available  in  the 
current  or  future  versions  of  the  ground  control  stations.  This  is  particularly 
helpful  when  one  of  the  parameters  reaches  a  dangerous  level.  The  bullet  graphs 
are  a  variation  of  traditional  bar  graphs,  with  additional  context  provided  by 
shading  different  thresholds  within  the  graph.  This  provides  not  only  the  current 
value  but  the  relation  of  that  value  to  important  thresholds  for  that  specific 
parameter. 


Fig.  4  Sparkline  and  bullet  graphs 

Among  the  new  graphs  that  were  created  is  the  flight  path  graph  (FPG).  The  FPG 
is  a  moving  cone-shaped  graph  of  relevant  information  in  the  immediate  path  of 
the  UAV  (Fig.  5).  This  integrates  5  of  the  6  critical  information  areas  into  one 
graph.  If  the  heading  of  the  UAV  changes,  the  contents  in  the  FPG  are 
dynamically  updated  to  show  the  operator  any  new  threats  or  items  of  interest  in 
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the  new  flight  path.  The  intent  for  this  graph  is  to  show  any  points  or  things  of 
interest  (e.g.,  targets,  terrain,  weather,  and  operational  threats)  that  are  in  the 
current  flight  path  and  within  a  reasonable  distance  away  from  the  UAV.  The 
FPG  also  contains  information  regarding  wind  speed  and  direction;  it  has 
transformed  the  UAV  operator’s  task  of  computing  a  head  or  tail  wind  from  an 
information-seeking  task  (gathering  wind  speed,  direction,  and  current  UAV 
heading  from  3  different  displays)  and  mental-computation  task  (placing  all  that 
information  in  short-term  memory  and  comparing  the  wind  direction  to  heading 
information)  to  a  simple  visual  perceptual  task  of  viewing  the  arrow  in  relation  to 
the  UAV  icon  on  the  FPG.  There  is  no  comparable  graphic  in  the  current  or  future 
ground  control  station  interface. 


Fig.  5  Flight  path  graph 

Another  newly  invented  graph,  the  flight  path  profile  graph  (FPPG),  complements 
the  FPG  by  displaying  much  of  the  same  information  from  a  profile  perspective. 
By  displaying  a  moving  map  from  a  side  perspective  with  the  centerline  as  the 
current  UAV  altitude,  we  find  that  the  relationship  between  the  UAV  and 
hazardous  terrain  or  obstacles  is  clear  and  highlighted  (Fig.  6). 
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Fig.  6  Flight  path  profile  graph 


The  final  graph  is  a  modified  bullet  graph  to  illustrate  the  spatial  relationships 
between  the  UAV  and  1)  takeoff  location,  2)  multiple  planned 
targets/destinations,  and  3)  landing  site,  along  with  showing  the  estimated  time  of 
arrival  to  target,  distance  to  empty,  fuel  status,  fuel  warning,  and  where  along  the 
planned  flight  path  fuel  may  reach  empty  (Fig.  7).  The  flight  path  is  transformed 
and  shown  along  a  linear  distance  path  where  the  bottom  of  the  graph  represents 
the  origin  point  where  the  UAV  took  off.  The  bar  itself  represents  the  distance 
flown,  and  the  various  bars  represent  planned  targets/destinations  showing  the 
relative  distance  between  each  other  and  to  the  UAV  itself.  The  triangle 
represents  the  landing  area,  which  may  or  may  not  be  the  same  as  the  takeoff 
location.  If  fuel  is  calculated  to  run  out  at  any  point  within  the  flight  plan,  a  red 
fuel  icon  is  displayed  and  moves  into  the  position  along  the  flight  plan  where  fuel 
would  expire  (Fig.  8).  Any  changes  in  the  flight  plan  will  trigger  the  fuel  alert  to 
be  recalculated  and  displayed. 


Fig.  7  Distance/fuel  graph 


Fig.  8  Distance/fuel  graph  with  fuel  warning 


3.2  Cognitively  Tailored  Interface 


The  first  of  several  planned  cognitive  skills  and  styles  (Fig.  1)  was  created  using  the 
wholistic-analytic  cognitive  style  dimension.  The  wholistic-analytic  cognitive  style 
continuum  describes  how  an  individual  organizes  and  structures  information.  The 
wholistic-analytic  scores  were  divided  into  3  equal  score  ranges  to  create  3 
categories  of  users  and  matching  interfaces:  1)  wholisitic,  2)  hybrid,  and  3)  analytic. 

The  wholistic  interface  emphasizes  information  at  a  global  level  and  displays 
more  details  only  when  an  alert  is  triggered  or  on  demand  by  the  user.  The 
wholistic  interface  emphasizes  the  FPG  because  of  its  ability  to  generate  the  “big 
picture”  of  the  current  flight  path  (Fig.  9).  Additional  details  are  hidden  until 
necessary  for  the  aircraft  status,  which  shows  the  sparkline  graphs  across  all  flight 
parameters  with  the  exact  value  and  its  corresponding  bullet  graph  hidden.  A 
simple  check  mark  is  provided  as  a  high-level  indicator  that  each  of  those  system 
parameters  is  within  normal  working  ranges.  The  details  are  only  shown  to  the 
user  when  that  parameter  reaches  a  dangerous  level  in  order  to  further  diagnose 
the  issue  or  if  the  user  wants  to  inspect  it  by  manually  clicking  on  that  parameter. 


Fig.  9  The  wholistic  cognitively  tailored  interface  version 

The  analytic  interface,  on  the  other  hand,  places  the  information  emphasis  on 
lower-level  details.  Figure  10  shows  the  analytic  version  with  the  bullet  graphs 
and  exact  aircraft  status  values  displayed  instead  of  the  more  generic  check  mark 
found  in  the  wholistic  version.  The  FPG  has  been  replaced  by  the  FPPG  based  on 
its  ability  to  better  support  the  detailed  table  information  found  under  it.  The  table 
provides  details  related  to  the  threats,  terrain,  targets,  and  weather  shown  in  the 
FPPG.  Additional  details  are  provided  for  the  fuel  status  in  the  form  of  2 
additional  bullet  graphs. 
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Fig.  10  The  analytic  cognitively  tailored  interface  version 

The  hybrid  interface  attempts  to  tailor  the  information  to  those  individuals  that 
fall  between  the  2  wholistic-analytic  extremes  by  providing  both  a  high-level 
overview  and  the  details  within  the  analytic  interface.  The  overview  is  provided 
by  the  inclusion  of  both  the  FPG  and  FPPG  and  only  gives  the  detail  views  of 
aircraft  status  when  there  is  an  alert  (Fig.  11).  Details  are  provided  with  the 
inclusion  of  the  table  that  complements  the  FPPG  data. 


Fig.  11  The  hybrid  cognitively  tailored  interface  version 
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3.2.1  Goodness-of-Fit  Experiment 

The  creation  of  the  CTIs  involves  3  primary  steps:  1)  tailoring  information  based 
on  the  targeted  cognitive  skill/attribute,  2)  testing  the  design,  and  3)  refining  the 
information  design.  A  testing  environment  was  created  to  test  the  designed  CTI 
based  on  user  performance  and  user  preference.  The  purpose  of  this  type  of 
experiment  is  to  refine  and  validate  the  design  generated,  which  may  take  several 
iterations  to  appropriately  match  how  the  information  is  presented  through  the  UI 
with  that  cognitive  attribute  of  the  individual. 

3. 2. 1.1  Goodness-of-Fit-Study  Overview 

The  study  presented  the  participants  with  3  types  of  interface  designs  to 
investigate  how  their  preference  varies  between  the  interface  designs  and  their 
measured  cognitive  style  along  the  wholistic  and  analytic  dimension.  The 
participants  were  presented  with  still  images  of  the  interface  and  asked  to  extract 
specific  information  from  it  to  gauge  their  preference  among  the  3  interface  types. 

Seventeen  college  student  volunteers  from  the  University  of  Alabama  at 
Huntsville  were  recruited.  All  responses  were  electronically  collected  via  mouse 
and  keyboard  inputs.  The  Extended  Cognitive  Styles  Analysis-Wholistic  Analytic 
test  (ECSA-WA)  (Peterson  and  Deary  2006)  is  a  cognitive  task  that  compares 
how  long  the  participant  takes  to  judge  the  similarity  of  2  shapes  (wholistic  task) 
with  how  long  the  participant  takes  to  judge  whether  a  particular  shape  is 
embedded  in  a  more  complex  shape  (analytic  task).  Participants  were  encouraged 
to  perform  the  test  at  their  own  pace,  and  the  stimuli  remain  on  the  computer 
screen  until  answered. 

3. 2. 1.2  Procedure 

The  interface  chosen  for  the  experiment  is  one  for  operating  UAVs.  The 
experiment  included  a  practice  session.  The  hypothesis  is  that  the  interface  that 
matches  the  person’s  cognitive  style  will  be  preferred  and  used  more  often  by  the 
user. 

Participants  went  through  a  short  eye-tracking  calibration  procedure  (5  min) 
where  they  looked  at  different  points  of  the  computer  screen  for  a  few  seconds 
each.  They  then  completed  the  ECSA-WA  on  a  computer.  They  were  introduced 
to  each  interface  with  instructions  on  how  to  read  the  instruments.  For  each  of  the 
3  interfaces  presented,  the  participants  were  asked  5  questions  to  provide  practice 
reading  the  interface.  All  3  interfaces  were  displayed  on  a  single  screen,  and  the 
participant  was  asked  20  questions  about  information  represented  in  the 
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interfaces.  The  order  and  placement  of  the  3  interfaces  were  randomly  presented 
for  each  question.  All  the  interfaces  displayed  the  same  data,  just  in  different 
ways.  Eye-tracking  data  was  used  to  detect  consistent  patterns  of  use.  This  session 
was  followed  by  a  short  interview  session  that  asked  their  subjective  preferences 
and  feedback  on  how  they  used  the  interfaces. 

3. 2. 1.3  Results 

The  results  from  the  ECSA  classified  the  majority  of  the  participants  as  having  an 
analytic  cognitive  style  (n  =  7,  50%),  followed  by  a  hybrid  cognitive  style  (n  =  5, 
36%),  with  the  least  amount  of  participants  classified  as  having  a  wholistic 
cognitive  style  (n  =  2,  14%).  The  response  accuracy  to  the  ECSA  was  very  high 
(96%).  The  student  volunteers  were  all  engineering  students,  which  explains  the 
high  rate  of  analytic  styles  and  low  rate  of  wholistic  styles  among  this  sample. 
Accuracy  in  the  CTI  survey  was  90%  for  both  the  performance  and  preference 
sections.  Participants  spent  an  average  of  14.15  s  ( SD  =  12.71  s)  on  each  of  the 
performance  questions,  and  10.57  s  (SD  =  5.86  s)  on  each  of  the  preference 
section  questions. 

Participants  completed  a  self-report  questionnaire  at  the  end  of  the  experiment  to 
report  which  interface  they  preferred.  The  questionnaire  revealed  that  71%  of  the 
participants  preferred  the  interface  that  matched  their  cognitive  style  scores.  Eye¬ 
tracking  data  gave  additional  insight  into  their  preferences  by  tracking  where  their 
attention  was  allocated  to  the  most  when  all  3  interfaces  were  presented.  Among 
the  15  participants  that  had  usable  eye-tracking  data  (e.g.,  Fig.  12),  50%  of  them 
spent  the  most  time  on  their  matching  interface.  These  results  suggest  that  the 
design  used  for  each  of  the  3  wholistic-analytic  cognitive  styles  matched  their 
individual  cognitive  styles  to  some  degree. 

The  feedback  and  performance  on  the  questions  provided  valuable  insight  that  led 
to  additional  tailoring  of  the  data  visualizations  to  each  of  their  respective 
cognitive  style  type.  These  changes  have  been  implemented  into  the  latest  version 
of  the  interfaces  for  future  testing  and  evaluation.  Additionally,  future 
experiments  will  address  the  unequal  distribution  of  cognitive  styles  represented 
in  the  experimental  sample. 
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Fig.  12  An  example  heat  map  showing  the  distribution  of  eye  gazes  for  one  participant 
across  the  3  cognitively  tailored  interface  designs  as  they  retrieve  information  to  answer  a 
question 

3.3  UAV  Experimental  Environment 

The  UAV  experimental  environment  will  be  used  to  measure  the  effect  of  the 
wholistic-analytic  CTI  in  addition  to  future  cognitive  styles  and  attributes  to  be 
added  to  the  system.  The  design  and  development  of  the  UAV  experimental 
environment  proved  to  be  the  most  ambitious  component  of  the  research.  The 
complete  experimental  environment  is  still  under  development.  The  experimental 
environment  consists  of  4  components: 

•  Cognitive  style  assessment  tool 

•  Customizable  UI  with  unique  data  visualizations 

•  Customizable  UAV  flight  simulator 

•  Embedded  human  performance  measures  for  UAV  operations 

The  cognitive  style  assessment  tool  consists  of  both  the  ECSA-WA  test  and  the 
Verbal-Imagery  Cognitive  Style  test  (Peterson  et  al.  2005).  Both  tests  have  been 
implemented  into  the  experimental  environment  so  the  scoring  is  automated  and 
immediately  available  for  the  system  to  select  the  matching  interface  design  based 
on  that  individual’s  cognitive  style/s.  Additional  testing  mechanisms  for  other 
cognitive  skills  and  levels  of  domain  expertise  can  be  added  to  the  system  to 
continue  expanding  its  scope. 
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The  customizable  UI  was  critical  to  the  experimental  environment  in  order  to 
present  the  user  with  different  data  visualizations  and  interface  configurations 
based  on  their  individual  cognitive  attributes.  The  flexibility  needed  within  the  UI 
prevented  the  use  of  any  commercial  off-the-shelf  products  and  drove  the 
development  to  a  completely  custom  system.  The  benefit  of  this  allows  for 
complete  control  over  the  UI  configuration  and  few  limitations  on  the  type  of  data 
visualizations  that  can  be  invented  and  experimented  with. 

The  UAV  simulation  allows  the  experimenter  to  customize  a  50-  to  70-min 
mission  consisting  of  10  legs  with  command  targets  at  the  end  of  each  leg  and 
based  off  of  a  similar  STE  developed  by  Wickens  and  Dixon  (2002).  Each  leg  is 
approximately  5-8  min  long  and  covers  about  10-12  km  in  distance.  An  Xbox 
controller  is  used  to  manipulate  the  aircraft  and  camera  view.  Within  the 
simulation,  a  map  view  and  a  camera  view  are  presented  (Figs.  9-11).  The 
command  target  is  located  at  the  end  of  each  mission  and  requires  the  operator  to 
loiter  around  it  once  it  is  detected  and  respond  to  questions  regarding  the 
command  target. 

As  mentioned  previously,  the  task  is  structured  around  the  3  critical  and 
fundamental  tasks  of  the  UAV  operator:  1)  navigation,  2)  target  detection,  and  3) 
monitoring  for  system  failures.  Navigation  is  done  by  entering  destination 
coordinates  and/or  manually  directing  the  UAV.  Target  detection  is  supported  by 
introducing  targets  of  opportunity  (TOO)  into  the  mission  legs.  A  TOO  is  a  square 
bunker  in  3  sizes  with  1-3  tanks  and/or  helicopters  surrounding  it.  There  is  one 
TOO  per  mission  leg,  which  is  randomly  located  in  the  middle  60%  of  the  screen. 
The  system  records  any  detections  and  reaction  times  for  detections.  The  final 
task  of  system  monitoring  is  done  by  simulating  system  failures  and  alerts.  Each 
system  failure  lasts  for  30  s  and  then  auto-resets  if  not  detected.  There  are  8 
system  failures  for  the  mission,  so  not  every  leg  has  one  and  no  leg  has  more  than 
one.  The  number  of  correct  detections  and  reaction  time  of  detection  is 
automatically  recorded  by  the  system. 

In  addition  to  these  performance  measures,  situation  awareness  is  assessed  by 
pausing  the  simulation  at  3  random  times  within  the  mission.  The  information  on 
the  screen  is  hidden  during  these  situation  awareness  segments,  and  4  questions 
about  the  state  of  the  system  are  presented  to  the  user  to  answer.  Their  accuracy 
and  response  time  is  recorded.  Workload  will  also  be  manipulated  creating 
different  workload  conditions. 
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4.  Conclusions 


This  research  project  has  attempted  to  expand  the  current  human  system  design 
approach  from  designing  for  the  masses  to  designing  for  the  individual.  The  value 
placed  on  information  will  only  increase  in  the  future.  The  better  able  we  are  to 
tailor  information  to  the  individual  needs  and  requirements  of  both  the  user’s  task 
and  the  individual  attributes  of  the  user,  the  more  power  and  knowledge  we  can 
provide  them.  This  still  remains  largely  an  unexplored  area  for  considerable  future 
research. 

In  attempting  to  create  a  CTI  for  the  UAV  domain,  new  ways  to  visualize 
information  to  the  UAV  operators  were  invented  that  are  based  primarily  on  the 
nature  of  the  task  and  what  we  know  about  the  psychology  of  perception.  A  CTI 
was  designed  and  implemented  with  preliminary  testing  conducted.  Additionally, 
a  research  paradigm  to  investigate  a  growing  repository  of  cognitive  attributes  and 
skills  was  created  along  with  the  tools,  measures,  and  experimental  environments 
needed  to  investigate  these  issues  further. 

Future  efforts  will  continue  with  the  experimentation  paradigm  developed  in  this 
project  (Fig.  1)  with  the  completed  UAV  experimental  environment  while  also 
working  with  the  unmanned  aerial  systems  operators,  developers,  and  community 
to  transition  the  designs  and  technology  that  can  improve  performance  for  UAV 
operators. 
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1733  PLEASONTON  RD  BOX  3 
FORT  BLISS  TX  79916-6816 

8  ARMY  RSCH  LABORATORY  -  HRED 
(PDF)  SIMULATION  &  TRAINING 
TECHNOLOGY  CENTER 
RDRL  HRT  COL  G  LAASE 
RDRL  HRT  I  MARTINEZ 
RDRL  HRT  T  R  SOTTILARE 
RDRL  HRT  B  N  FINKELSTEIN 
RDRL  HRT  G  A  RODRIGUEZ 
RDRL  HRT  I  I  HART 
RDRL  HRT  M  C  METEVIER 
RDRL  HRT  S  B  PETTIT 
12423  RESEARCH  PARKWAY 
ORLANDO  FL  32826 

1  ARMY  RSCH  LABORATORY  -  HRED 
(PDF)  HQ  USASOC 

RDRL  HRM  CN  R  SPENCER 
BLDG  E2929  DESERT  STORM  DRIVE 
FORT  BRAGG  NC  28310 
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1  ARMY  G1 
(PDF)  DAPE  MR  B  KNAPP 

300  ARMY  PENTAGON  RM  2C489 
WASHINGTON  DC  20310-0300 

ABERDEEN  PROVING  GROUND 

12  DIR  USARL 
(PDF)  RDRL  HR 

L  ALLENDER 
P  FRANASZCZUK 
K  MCDOWELL 
RDRL  HRM 

P  S  A V AGE-KNEPS  HIELD 
RDRL  HRM  AL 
C  PAULILLO 
RDRL  HRM  B 
J  GRYNO VICKI 
RDRL  HRM  C 
L  GARRETT 
RDRL  HRM  DI 
J  HANSBERGER 
RDRL  HRS 
J  LOCKETT 
RDRL  HRS  B 
M  LAFIANDRA 
RDRL  HRS  D 
A  SCHARINE 
RDRL  HRS  E 
D  HEADLEY 
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